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EMBEDDING INFORMATION IN A DIGITAL IMAGE DIGITIZED 
FROM A DEVELOPED PHOTOGRAPHIC FILM 
CROSS REFERENCE TO RELATED APPLICATIONS 

Reference is made to the commonly-assigned U.S. Patent 
Application Serial No. 09/339,755, filed June 24, 1999, entitled "Recording 
Invisible And Detectable Information In Photographic Images" by Xin Wen, and 
commonly-assigned U.S. Patent Application Serial No. 

ZQQAff) filed concurrently herewith entitled "Embedding Information in a Printed 
Image", by Xin Wen et al, the disclosures of which are incorporated herein by 
reference. 



This invention relates to producing digital mages from scanned 
photographic film. 



remember their memories of past experiences. Photographic images can be 
captured by a photographic film or a solid state image sensor. Film-based 
photography is an analog process. An optical system forms an image of a scene 
on a film coated with a photosensitive material, most commonly silver halide 
emulsion grains. The exposure to photons activates a photochemical process in 
the silver halide emulsion grains, producing a latent image of the scene on the 
film. A latent image carries the information of the scene image, but it is not 
visible to human eyes. Subsequent development of the film containing the latent 
image can produce a visible image formed by dye molecules. The dye image on 
the photographic film can then be reproduced on a photographic paper. 
The exposed photographic films can be developed either in a central location, 
typically referred as a Central Lab, or a distributed site, which can be called as 
either a Minilab or a Microlab depending on the size and the throughput of the 
site. The traditional development and printing processes of the silver halide films, 
as described above, all involve wet chemical and analog processes. With the 
advancement of computer and imaging technologies in recent years, a demand has 



FIELD OF THE INVENTION 



BACKGROUND OF THE INVENTION 



Photography is a popular method for people to record and 




been created to provide users digital photographic services. By using digital 
methods, photographic images can be more readily received from and provided to 
users in various digital formats. 

One problem that has arisen from the usage of digital images is the 
detection of the unauthorized use of copyrighted or personal images. It is known 
that the professional photographers have been greatly concerned about the 
illegitimate use of their copyrighted images in the recent years. As the use of 
Internet and digital imaging devices become more popular, this concern will 
spread to the common consumers. 

SUMMARY OF THE INVENTION 

An object of the present invention is to provide high quality digital 
images scanned from developed photographic film which have embedded 
information that is invisible when printed on a receiver. 

This object is achieved by apparatus for embedding information in 
a digital image digitized from a developed photographic film in response to a film 
property and information to be embedded into the digital image, comprising 

a) a scanner for scanning the image on the developed 
photographic film to produce the digital image; 

b) means for storing the film property; 

c) input means for storing information associated with the 
image and to be embedded in the digital image; and 

d) means responsive to the stored film property and the 
associated information for modifying a predetermined number of pixel values with 
the associated information in the digital image so that the associated information is 
embedded in the digital image, wherein the embedded information can 
subsequently be extracted. 

An important feature of the present invention is that the 
information embedding is particularly suitable for use in existing photographic 
processes such as provided by Microlabs, Minilabs and Central Labs. 




# • 

-3 - 

An advantage of the present invention is that the high quality of the 
developed photographic image is preserved in the digital images even though it 
contains embedded information. 

Another advantage of the present invention is that the amount and 
the reliability of the embedded information can be optimized depending on the 
film property. 

A further advantage of the present invention is that the 
embedded information can be varied according to the user's input, thereby 
permitting adding personal messages and authenticating information. 

BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 shows an apparatus for embedding information in digital 
images scanned from developed photographic film, the information embedding is 
dependent on a film property and the user input in accordance to the present 
invention; 

FIG. 2 shows the detailed components of the computer of FIG. 1; 
FIG. 3 is a flow diagram of the operations of the apparatus of FIG. 

1; and 

FIG. 4 shows a curve of mean-square deviation of the optical 
density as a function of the speed of a photographic film. 

DETAILED DESCRIPTION OF THE INVENTION 
The present invention relates to an apparatus for embedding 
information in digital images scanned from developed photographic films. One 
important feature of the present invention is that the information embedding is 
conducted in response to a film property and the user input. In the present 
invention, the term "film property" includes one or more of the following 
properties: film speed, film type (e.g. 35 mm, APS, negative or positive, and 
black and white), APS format (Classic, panoramic, HDTV), or exposure 
conditions. The information is embedded into the digital images according to the 
film property so that the high photographic image quality is preserved in the 
digital images even with the embedded information. The film property is also 
used for maximizing the amount and the reliability of the embedded information. 
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In the present invention, the term "quality" or "image quality" refers to the image 
properties that define the visual effects of an image and the fidelity of the 
reproduced image compared to the original image. Such image properties can 
include grain noise, tone scale, color reproduction, sharpness, and the severity of 
image artifacts, which are well known in the imaging field. 

In accordance with the present invention, the more noise a film- 
based image has, the more data can be embedded into the image without affecting 
the quality of the image. That means that more data can be embedded in an image 
developed from a high-speed film than from a lower speed film. 

FIG. 1 shows an apparatus 1 0 in accordance with the present 
invention. The apparatus 10 includes a computer 20, film property extraction 
device 25 for extracting one or more film properties as described above, a film 
processor 30, a film scanner 40 and a digital printer 50. Although not shown, the 
apparatus 1 0 can also include an optical printer for printing photographic images 
on a photographic paper in a wet chemical process. 

The computer 20, shown in FIG. 2, includes a Central Processing 
Unit (CPU) 31, input device 32, such as a keyboard, and a mouse that receives 
user input information from the user 60, a monitor 33 for displaying user input 
information or authentication images, a memory 34 for storing digital images and 
user input information, and an image processor 35 for processing the digital 
images to embed information and conducting other image processing operations 
well known in the art. 

Referring again to FIG. 1 , a camera 70 records latent images of 
scenes, as described above. In the present invention, the term "scene" refers to 
any object that can be captured as an image on a photographic film. The film 
carrying the latent images is typically contained in a light sealed exposed film 
cassette 80 that is received by the film processor 30. 

As is well known in the photographic field, the film processor 30 
processes the film carrying the latent images in the exposed film cassette 80 in a 
wet chemical process. The exposed film is sequentially processed in several 
processing solutions: a developer solution, a bleach solution, a fixer solution, a 





-5- 



washing solution, or a combination of bleach/fix solution and wash/stabilizer 
solution. Finally, the developed film is dried in a drying box. In the present 
invention the film processor 30 can also use a dry process in which the digital 
images can directly be scanned from the films. The developed films 90 and 100 
5 are sent to the film scanner 40 or an optical printer (not shown) for printing on a 
photographic paper. 



pixel-wise fashion. The optical densities of the analog photographic image are 
read at each pixel location, producing a digital image file. The digital image is a 

10 representation of its analog images on the developed film 90 and 100 and can be 
conveniently stored on the electronic memory 34. Digital images can be easily 
transferred and shared over the modern information channels such as the Internet. 
The digital image can also be printed by the digital printer 50 in a pixel- wise 
fashion to produce a photographic image on a receiver. Although not shown, 

1 5 another scanner can be included for digitizing images on a receiver. The digital 
image can also be provided by a digital camera 1 10, or an optical disk 120 such as 
Kodak's Picture CD or Kodak's Photo CD that is read by a CD drive 130. 



digital images and embeds information into the digital images obtained by the film 
20 scanner 40. In the present invention, the embedded information is typically 

information associated with the image. In FIG. 1, different types of embedded 
information are represented by a "key" symbol. For example, the authentication 
information can be used to deter counterfeiting and illegitimate use. 



25 the information selected by the user, and the sound information. The 

authentication information can include a watermark, copyright, and the name of 
the user or the photographer. In the present invention, the term "user input 
information" refers to the information selected or produced by the user 60 and 
input to the computer 20. The user input information can include the user's 

30 identification, the time, location, background of scene etc. The embedded 

information can also include sound information such as voice and music. The 



The film scanner 40 scans the developed films 90 and 1 00 in a 



The image processor 35 in the computer 20 then processes the 



The embedded information can include authentication information, 
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sound information can be recorded at the scene of the image capture or at the 
Microlab or Minilab, or can also be selected from an electronic memory. 

The embedded information can be input to the computer 20 at the 
Microlab, Minilab or the Central Lab; it can also be produced or input from a 
5 remote location 140 such as a house through an information channel 150 such as 
the Internet. In FIG. 1, the "key" symbol next to the information channel 150 
represents that the embedded information can be transferred in a secure fashion. 

FIG. 3 illustrates the flow chart of the operations of the apparatus in 
accordance with the present invention. The developed film in the developed film 

10 cassette 90 is scanned in box 300. A digital image I(x,y) is produced as a result. 
The film properties are extracted by the film property extraction device 25 in box 
310. The film property 320 is produced in box 320. The film property 320 can be 
extracted from the magnetic coating layer on the APS film before the development 
of the exposed photographic film, or from the optical coding on the non-image 

1 5 areas of the photographic film after it is developed. The extracted film properties 
are stored in the memory 34 in the computer 20, which is accessible by the image 
processor 35 for information embedding. As described below, the parameters for 
embedding information in the digital image I(x 5 y) are adjusted in box 330 
according to the film property 320. The user 60 produces or selects information in 

20 box 340. The user input information is also stored in memory 34. The user input 
information is embedded to the digital image I(x,y) using the adjusted embedding 
parameters in box 350 to produce a new digital image V(x,y) that carries the 
embedded information. 

The information is embedded into the digital image I(x,y) using 

25 parameters adjusted depending on the film property 320 so that the high image 
quality in the digital image I(x,y) (derived from the photographic film) is 
preserved in the digital image T( x ?y) with the embedded information. The film 
property 320 is also used for maximizing the amount and the reliability of the 
embedded information. For example, the amount of data embedded can be 

30 maximized according to the film speed without affecting the image quality. Films 
of higher speed have less light for exposure in a shorter period of time. In the 



? 




-7- 



photographic films, the silver halide emulsion grains acts like photon detectors. 
The high speed films use larger emulsion grain to absorb the increasingly sparse 
photons, which tends to produce more granularity (or fluctuations in optical 
densities) in the developed film. 
5 FIG. 4 is a graph showing that as film speed is increased, the 

standard deviation of the photographic noise increases. In the trade of 
photographic technologies, it is also well known that film granularity is also 
dependent on the film type: 35 mm, APS, negative or positive, and black and 
white, as well as exposure conditions. 

10 In accordance with the present invention, the more noise a film- 

based image has, the more data can be embedded into the image without affecting 
the quality of the image. That means that more data can be embedded in an image 
developed from a high-speed film than from a lower speed film. 

The visual impact of changes in an image is documented and has 

1 5 been facilitated in computer models of the human visual system (HVS). A 

practical and well-known model is the contrast sensitivity function (CSF), which 
relates to the signal contrast that is required for detection as a function of spatial 
frequency, noise, and other parameters. A basic description of the CSF can be 
found in chapter five of "Foundations of Vision, " by Brian A. Wandell, Sinauer 

20 Associates Inc., Sunderland, MA, 1995. This reference also includes CSF plots 

based on measured sensitivities. A number of CSF models have been described in 
the literature, and in a preferred embodiment of the present invention, specific use 
is made of a model described by Daly in "Application of a noise-adaptive contrast 
sensitivity function to image data compression" Optical Engineering, Volume 29, 

25 number 8, pages 979-987, August 1990. This CSF model shares characteristics 
similar to the experimentally measured data, e.g., the sensitivity decreases as an 
increasing function of both horizontal and vertical spatial frequency and 
background noise level. The exact form of the CSF derived from this model is 
dependent upon a number of viewing condition parameters, including viewing 

30 distance, adaptation light level, noise level, wavelength, and chromatic channel. 
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By adjusting the amplitude of the embedded signal in accordance with the CSF, 
we can maximize the amount and quality of information embedded. 

The information embedding in the digital images I(x,y) in the 
present invention is determined by the stored film property 320 in memory 34 and 
the user input information 330. In the data embedding, a predetermined number of 
pixel values in the digital image is modified. The preferred algorithms in 
accordance with the present invention for forming non-visible information 
(message) in an image may be broken down into two parts, the process of 
embedding the message into an image and the process of extracting the message 
after the image has been used. 

In the embedding part, the message (the user input information 
330) and the carrier are convolved to form a scrambled signal. The scrambled 
signal then is scaled to ensure that the amplitude of the resultant scrambled 
message is not visually perceptible. Finally, the scrambled message is added to 
the image. The specific form of the convolution operation is referred to as circular 
convolution (see "Digital Signal Processing 55 by Alan V. Oppenheim and Ronald 
W. Schafer, Prentice-Hall, Inc., pp-101-1 15) which can be efficiently performed in 
either space domain (image domain) or Fourier frequency domain representations. 
The process of data embedding can be mathematically described as 

/'(x, y) = /(*, y) + a(M(x, y) * C{x 9 y)) 

where I(x,y) denotes the digital image digitized from the developed photographic 
film, I'(x,y) denotes the final image containing the embedded data, M(x 9 y) 
denotes the message that is to be embedded in the digital image I(x 5 y), C(x,y) 
denotes the carrier signal, M(x,y)*C(x,y) denotes the scrambled image, a is a 
scale factor, and * denotes the process of circular convolution. The scale factor 
a is chosen to be small enough so that the message is visually undetectable. That 
is, the value of a is chosen in such a way that the maximum amplitude of the 
embedded signal does not exceed a pre-specified threshold. This threshold can be 
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obtained by calculating the contrast associated with the CSF associated with a 
given film at typical viewing conditions of the final image. 

Now, typically, the CSF is represented in the Fourier domain. This 



5 should be pre-calculated. If the maximum value of the added information in the 
Fourier domain is always less than or equal to the contrast value associated with 
CSF, then the embedded signal will be invisible. The value a , for which the 
maximum amplitude of the quantity (M(x,y) * C(x,y)) is obtained can be found 
through simple iterative means by starting with a very small value of a and 
10 incrementing it until the first value of the Fourier transform of the embedded 

message becomes the same level of as a point of in the contrast level associated 



used as input for the extraction. Or alternatively, the image on a receiver must be 
1 5 first scanned and corrected for rotation and scale to produce a digital image 

analogous to F( x 5 y)- For maximum performance, the scanner should be set to a 
resolution high enough not to alias any of the hidden data. If the scanner induces 
an MTF loss (that is, the scanner attenuates some frequencies), a suitable 
compensation sharpening filter is preferably applied. The preferred method for 
20 correcting rotation and scale is disclosed in Honsinger, et al., "Method for 

detecting rotation and magnification in images", U.S. Patent 5,859,920. After this, 
the scanned image I'(x 9 y) is cross correlated with a replica of the carrier, 
C(x,y) , as described by the following: 



where ® denotes the operation of circular cross correlation. 

To see how this recovers the message, replace I'{x,y) with the 
embedding equation above to yield: 



implies that the Fourier transform of the added message (M(x,y) * C{x,y)) 



with CSF. 



The digital image F(x,y) carrying the embedded information can be 
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M\x,y) = [l(x,y) + aM(x,y) * C(x, y))<8>C(x,y) = 
= 7(x, y) <8> C(x, y) + aM(x, y) * [C(x, y) ® C(x, y) J 



Term 1 Term 2 



The above relationship simply states that the recovered message consists of the 
sum of two components, one term is the cross-correlation of the digital image 
5 I(x 5 y) with the carrier signal, and the second term is the autocorrelation of the 

carrier signal convolved with a scaled version of the digital message I(x,y). Using 
a random carrier that is substantially uncorrelated with the digital image I(x,y) can 
minimize the effect of the first term. From examining the second term, it is clear 
that the resolvability of the recovered message is affected by the autocorrelation of 

10 the carrier signal C(x,y) . Ideally, to achieve maximum resolvability, the 

autocorrelation of the carrier signal should be a delta function, S(x,y) , which 
from Fourier analysis theory is equivalent to suggesting that the carrier signal 
should contain all frequencies at equal amplitude. A carrier with these properties 
is disclosed in Honsinger, et al. in the Patent Application Serial No. 08/848,1 12, 

1 5 "Method For Generating An Improved Carrier For Use In An Image Data 
Embedding Application". 



embedding in box 350 in the digital image I(x,y) can be implemented redundantly 
in a plurality of tiles in an image. The redundancy of the embedded information 

20 increased the reliability and signal-to-noise ratio when the embedded information 
is extracted. The number and the size of the tiles used in the embedding and 
extracting information are dependent on the film property 320 where the digital 
image I(x ? y) is developed from. For example, the 35 mm film and different APS 
formats (Classic, panoramic, HDTV) have different film frame dimensions and 

25 aspect ratios, as well as different granularity levels. The sizes and aspect ratio of 
the tiles used for information embedding in an image developed from these films 
are optimized to provide the most redundancy and thus reliability, and the highest 
signal-to-noise ratio. 



In accordance with the present invention, the information 
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The embedding and extracting information to and from the images 



I(x,y) digitized from the developed photographic films can also include techniques 
disclosed by Daly, et al. "Method for embedding digital information in an image" 
in U.S. Patent 5,859,920, Honsinger, et al. in "Method For Detecting Rotation 
5 And Magnification In Images" in U.S. Patent 5,835,639, and Honsinger, et al. in 
the Patent Application Serial No. 08/848,1 12. Alternatively, the algorithms 
disclosed in Rhoads in U.S. Patent 5,710,834, "Method And Apparatus 
Responsive To A Code Signal Conveyed Through A Graphic Image", can be used 
but the amount of information that can be embedded may be too limiting for some 
10 applications. These disclosures are herein incorporated. 



to the digital printer 50 for printing printed image 180. The digital images with 
embedded information can also be sent to a CD writer 200 that writes these digital 
images on a writeable optical disk 210 such as a CD-R. The digital images I'(x,y) 

1 5 can also be stored on other similar storage media such as a solid-state memory 

card. The printed image 180 and the digital images stored on the optical disk 210 
contain the embedded information that can be extracted. Depending on the type of 
the embedded information as described above, the extracted information can be 
used to authenticate the owner of the image, or used to describe information 

20 related to the capture of the image (time, location, occasion etc.), or used to play 
sound information associated with the image. 



to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 



The digital images T(x,y) with embedded information can be sent 



The invention has been described in detail with particular reference 
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PARTS LIST 
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optical disk 
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140 


remote location 




150 


information channel 




180 


printed image 




200 


CD writer 




210 


optical CD 




300 


scanning the developed film 
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